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DIJITAL TEKNOLOJIi TEMELLI YABANCI DIL OGRETIMININ KELIME 
OGRENIMI UZERINE ETKISi: BIR META-ANALIZ CALISMASI 


Sayeed Masood HAIDARI’, Seda BAYSAL?, Sedat KANADLF? 

OZ 

Diital teknoloji destekli yabanci dil kelime 6grenimi cevrimigi ve ¢evrimdisi sézliikler, multimedya, sosyal medya, 
televizyon, bilgisayar, akilli telefonlar, tabletler ve mobil cihazlar gibi yeni ve modern teknolojilerin kullanilmasiyla 
kelimelerin 6grenilmesi anlamina gelmektedir. Alanyazin incelendiginde, dijital teknolojinin yabanci dil kelime 6greniminde 
etkisi oldugunu belirten galismalar oldugu gibi kelime 6greniminde etkisi olmadigini gésteren ortaya koyan ¢alismalar da yer 
almaktadir. Bu nedenle, bu ¢alismanin amaci dijital teknolojiye dayali 6gretimin yabanci dil kelime 6grenimi tizerindeki 
etkisini inceleyen Tiirkiye’de yapilan deneysel ¢alismalarin meta-analizini yaparak genel etki biiyiikliinti hesaplamak ve bu 
etki biiyiikligiiniin ¢esitli degiskenler agismdan anlamli bir bigimde farklilasma gésterip géstermedigini belirlemektir. Bu 
arastirmada meta analitik tarama yontemikullanilmistir. 2003-2018 yillar1 arasinda dijital teknolojiye dayal 6gretimin 
yabanci dil kelime 6grenimi tizerindeki etkisini inceleyen kodlama Olctitlerine uygun 34 galigsma (41 karsilastirma) 
belirlenmistir. Yapilan meta analiz sonucunda, dijital teknolojiye dayali 6gretimin yabanci dil kelime 6grenimi tizerinde 
pozitif, genis aralikta bir etki biiyiikliigiine sahip oldugu tespit edilmistir. Bu etki bityiikliigiintin galismanin tirtine gére 
tezlerin lehine ve gruplama yéntemine gore rastgele atama lehine anlamli farklilik gésterdigi; ancak ¢alismanin kalitesine, 
6gretim kademesine, kullanilan teknoloji tiirtine ve uygulama stiresine gore anlamli fark géstermedigi tespit edilmistir. 


Anahtar Kelimeler: kelime 6grenme, yabanci dil, dijital teknolojiler, etki biiytikligii 


THE IMPACT OF DIGITAL TECHNOLOGY-MEDIATED FOREIGN LANGUAGE 
INSTRUCTION ON VOCABULARY LEARNING: A META-ANALYTIC REVIEW 


ABSTRACT 


The digital technology-mediated foreign language vocabulary learning involves the use of technological tools such as online 
and offline dictionaries, multimedia, social media, television, computer, smart-phones, tablets, and similar others. Although 
many research findings in the existing literature have emphasized the effectiveness of these technological tools on foreign 
language vocabulary learning, some studies also have suggested that the use of these tools in vocabulary learning does not 
make a significant difference in student achievement. This meta-analytic research aimed to investigate the overall impact of 
digital technology-mediated instruction on learning foreign language vocabulary in the Turkish context by calculating the 
mean effects followed by some moderator analyses. Therefore, 34 experimental studies (41 comparisons), conducted between 
the years 2003-2018, were included in the study according to the pre-defined coding criteria. The findings revealed that 
incorporation of digital technologies in foreign language teaching has a positive large impact on students’ vocabulary 
learning achievement. The effect sizes of two moderators, the study type and grouping methods, were significant, while the 
study quality, education level, type of technology, and treatment duration, were not significant moderators. 
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1. INTRODUCTION 


Vocabulary learning has been considered as a basic and significant part of foreign language learning for long 
years. Various researchers acknowledged that vocabulary learning is a fundamental component of foreign and 
second language competency (Bozavli, 2017; Cellat, 2008; Harley, 1996; Nation, 2001; Read, 2000). Krashen 
(1989) indicated that a large part of the meaning in a language is largely eventuated by words and hence, lack of 
vocabulary is regarded as the biggest obstacle in using the target language in an efficient way. Therefore, 
vocabulary is an essential factor that attaches four basic skills such as listening, reading, speaking and writing 
with a view to mastering a language (Kayaoglu, Akbas ve Oztiirk, 2011; Nam, 2010). As noted by Wilkins 
(1972), “without grammar very little can be conveyed, [but] without vocabulary nothing at all can be conveyed”. 
Thus, learning vocabulary with effective and innovative methods plays a significant role in foreign language 
education (Cited in Mahdi, 2018, p. 136). 


The status of vocabulary learning may largely be attributed to the language teaching approaches which vary 
depending on the time. Because there exists a close relation between vocabulary learning and foreign language 
learning methodologies (Ketabi & Shahraki, 2011). Considering the methodologies in language teaching, Celce- 
Murcia (2001) divided the major trends into nine approaches, such as, (1)Grammar-Translation, (2)Direct, 
(3)Reading, (4)Audiolingualism, (5)Oral-Situational, (6)Cognitive, (7)Affective-Humanistic, (8)Comprehension- 
Based, and (9)Communicative. The common roles of these approaches in vocabulary learning have based upon 
the fact that language learners are supposed to prepare lists of vocabulary and memorize them, and that 
vocabulary could be learned without any effort. Besides, it has been generally presumed that vocabulary can take 
care of itself and that there is no need for direct vocabulary instruction (Schmidt, 2000). 


However, the perspective towards vocabulary learning has radically changed in the 21“ century, and outpouring 
interests have been demonstrated in this field. So to speak, the mobility towards efficient methodologies for 
teaching and learning vocabulary have emerged; in addition, language experts and researchers have 
recommended numerous strategies and techniques with regard to vocabulary learning, signifying the efforts of 
the foreign language learners (Ketabi & Shahraki, 2011; Mizumoto & Takeuchi, 2009; Nation, 2001). In this 
direction, two major approaches regarding vocabulary teaching and learning have been put forward (Ketabi & 
Shahraki, 2011; Nation, 2001). The first approach is the incidental (implicit) learning that focuses on acquiring 
vocabulary through four basic skills such as listening, reading, speaking, or writing. It also refers to the fact that 
learners are supposed to encounter a certain word several times in different contexts in order to acquire it 
accurately. On the other hand, the second approach is the intentional (explicit) learning which constitutes 
deciding the words that learners must learn, presenting the new words as well as developing fluency. It also 
involves processes such as a demonstration, a picture or a real object (Gass, 1999). Namely, both directions are 
expected to increase the vocabulary knowledge that language learners need to have so as to ensure effective 
communication. 


By the same token, vocabulary learning can emerge in two different environments, namely, technology-based 
and traditional environment (Mahdi, 2018). Technology-based environment points to the use of new and modern 
technologies for vocabulary learning such as online and offline dictionaries, multimedia, social media, television, 
computer, smart-phones, tablets, and mobile devices. Therefore, it may be wise to mention that fast-moving 
digital technology provides learners in the vocabulary learning area with limitless opportunities. Thanks to the 
innovative methods and materials that digital technology presents, vocabulary learning can be much more 
colorful, supportive and motivating for the learners during the learning process. Whereas, the traditional 
environment is to use the word cards, dictionaries, word lists etc. Hence, technology-based environments have 
superior characteristics compared to the conventional one, in that learners can associate the meanings of the 
words with the images, sounds; meanwhile, they are exposed to the pronunciation and the written form of words 
simultaneously. 


1.2. Technology-Mediated Vocabulary Learning 


Digital technology today has a remarkable position in diversifying the foreign language education system like 
any other fields. Numerous technological devices are being ubiquitously integrated into the teaching and learning 
process. The internet and computer technology as the main tools alongside many other digital technologies have 
facilitated foreign language learning via virtual learning environments making it unrestrictive to time and space 
barriers (Baturay, Yildirim,& Daloglu, 2007). Digital technology has been in use for a few decades in language 
education. However, with the maturation of technology new sophisticated tools able to connect to the internet are 
being introduced and utilized widely in foreign language education. For example, online and offline dictionaries, 
language learning websites, video games, language applications, weblogs, wikis, multimedia, social media, TV 
programs, chat rooms and tools like tablets, smart-phones and iPods are some amongst the many used by the 
language learners around the world (Golonka, Bowels, Frank, Richardson & Freynik, 2014; Richards, 2015). 
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These sources not only provide extra opportunities to the learners to practice language skills but also facilitate 
vocabulary enrichment in the target language. It is obvious that stronger vocabulary means stronger language 
skills and the effective use of the target language. Although it makes the main component of language 
proficiency, most of the focus has been on developing other aspects of a language. One of the underlying 
problems facing individuals in foreign language learning could be overemphasizing the importance of language 
skills and grammar ignoring that vocabulary builds the basis to meet those purposes. In other words, vocabulary 
is an integral part of language learning process and has a pivotal role in building on language proficiency than 
grammar does, “be it a mother tongue or the second/foreign language” (Kilickaya & Krajka, 2010, p. 81). 


However, the integration of modern technology into language education has made vocabulary learning easy and 
motivating. Learners, for instance, can get instant translation and definitions for unknown vocabulary or 
expressions with plentiful examples through diverse technological tools. So doing, they become aware of 
divergent usages of a word in a matter of seconds or minutes and also get motivated to acquire new vocabulary 
in the process. In addition, accessing materials and applications related to foreign language learning over the 
internet through computers, smart-phones, tablets etc, has made language acquisition easier by reading, writing, 
watching, listening and even speaking with the other language learners or natives. Students, for instance, could 
better learn the foreign language vocabulary through animations, graphic materials, and games designed 
according to their age groups under the guidance of an adult. These and other sources will help both incidental 
and intentional vocabulary acquisition with lasting effects easing the developmental process of linguistic skills in 
return (Golonka et al., 2014; Richards, 2015). Richards (2015) maintains that such resources provide meaningful 
language learning opportunities for the students without restricting their learning to the in-class instruction. 


A great number of experimental and non-experimental studies are conducted throughout the world to examine 
the effectiveness of diverse digital technologies in foreign language vocabulary learning and acquisition and its 
superiority to traditional approaches considering different age groups and grade levels from school to university 
(e.g. Yip & Kwan, 2006; Fageeh, 2013; Shoaei & Alavi, 2016; Nejati, Jahangiri, & Salehi, 2018). Turkish 
researchers also have been examining the impacts of digital technologies in foreign language vocabulary 
learning since years proving their effectiveness over conventional classroom instruction (e.g. Fidan, 2003; Eren, 
2015; Giivendir & Gezgin, 2015; Kurt & Bensen, 2017; Ozer & Kocoglu, 2017). In contrast, there exist some 
contradictory findings, too, reporting that no significant difference exists between the instruction with or without 
technology in foreign language vocabulary learning (Tokag¢, 2005; Erkus, 2008; Basoz & Can, 2016). In order to 
deal with such contradictions in the existing research, a synthetic analysis is required to get a fuller picture of 
how effective digital technologies are in foreign language learning in this country. 


Only one meta-analysis study was found that explored the effectiveness of computer technologies on foreign 
language teaching against the conventional approaches in Turkey. The findings revealed that such tools have a 
significant impact on overall foreign language learning yielding a total effect size of 1.4321 (Tomakin & 
Yesilkurt, 2013). One more study was found on technology-assisted foreign language vocabulary learning on 16 
studies of different countries in the world (Mahdi, 2018). This particular study examined the effectiveness of 
using mobile devices against the conventional style in vocabulary learning. The results indicated that the use of 
mobile phones had a superior role in effective vocabulary learning than did conventional techniques. 


Despite the existence of numerous studies on foreign language vocabulary learning using technology, no meta- 
analytic research was found in this regard in Turkey. Therefore, this study aims to examine the effects of a 
variety of digital technologies used in foreign language vocabulary learning and acquisition synthesizing primary 
research findings in Turkish context through a meta-analysis research design to provide an awareness as regards 
the general role of digital technologies revealed in the existing studies conducted in around two decades, that is, 
within the years 2000-2018. As a matter of fact, these years saw evolutional changes in the advancement of 
digital technologies in human life influencing the functionality of different sectors in a positive way despite 
many shortcomings may also have existed in terms of availability and affordability. Nevertheless, the vitality of 
technological tools in the education paradigm cannot be ignored, especially in foreign language education. 
Therefore, it is worth to give a fuller picture of a well-researched area through research synthesis to determine 
and direct the focus of future research. Considering the discussion made above, the following question was asked 
in compliance with the purpose of the study at hand: 


What is the impact of digital technology-mediated instruction on learning foreign language vocabulary in the 
Turkish context? 
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2. METHOD 


2.1. Research Model 


This research has employed a meta-analytical review method in order to combine the results of the independent 
experimental studies. The meta-analytical review is a quantitative methodology for strengthening the growth of 
published research to more scientifically and comprehensively synthesize bodies of research (Chalmers, Hedges 
& Cooper, 2002). Cooper (1998) has indicated five stages in the process of meta-analysis: (i) problem 
formulation, (ii) data collection, (iii) data assessment (inclusion and exclusion criteria, (iv) data analysis and 
interpretation, and (v) presentation of the findings. Considering the five stages, Card (2012) has outlined four 
sections of a meta-analytical review: 


(i) Introduction provides a background in theory, methods, and prior findings as well as the aim of the studies, 
research questions, and hypotheses of the meta-analytic review. 


(ii) Method explains the research process with enough detail including literature search procedures, inclusion 
and exclusion criteria of the studies, coding of study characteristics and effect sizes along with data analytic 
strategy. 


(iii) Results refer to the accurate and clear report of the findings of the analyses. A much more conceptual or 
methodological organization of the results consists of descriptive information, central tendencies, and 
heterogeneity, moderator analyses, diagnostic analysis. 


(iv) Discussion: includes four components such as the review of findings, explanations, and implications of the 
findings, limitations, and conclusions. 


2.2. Literature Search Procedure 


The current literature search has been conducted through the use of various databases as follows: Educational 
Resources Information Centre (ERIC), ULAKBIM, EBSCO, and YOK Thesis Archive, between September 
2017 and December 2018. Final search has been carried out in January 2019. The research has applied the 
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following key terms in both Turkish and English: “vocabulary learning”, vocabulary acquisition”, “vocabulary 
learning and technology”, “digital technology and vocabulary learning”, “digital technology-assisted vocabulary 
learning”, “digital technology-assisted vocabulary teaching”, “vocabulary development” and “learning 
vocabulary through technology”. The related literature has been searched through examining the references of 
the collected studies. Access to the necessary data has been ensured by communicating with the authors of 
restricted access theses. In addition, the survey process has been carried out by two researchers, and the relevant 
studies have been selected by deciding on whether the studies found by the researchers are similar. As a result, 


the researchers had 267 studies regarding digital technology assisted vocabulary learning in Turkey. 


2.3. Study Inclusion and Exclusion Criteria 


Within the context of this study, the specified criteria determined for the inclusion of the studies in this meta- 
analysis study are as follows: (i) They must be carried out in Turkey between the years of 2003 and 2018, (ii) 
They must be articles, thesis (master or doctoral) or proceedings that employed experimental design, (iii) The 
studies must examine the effect of digital technology-assisted teaching on students’ vocabulary learning, (iv) 
They must include the duration time of the experiments. (v) Each study must report the quantitative data required 
to calculate impact sizes (sample size (NV), mean score (x), standard deviation (sd), t-test and p-value, r-value). 
(vi) They must employ parametric tests (t-test, F-test or ANOVA). A considerable number of studies were 
excluded from the current meta-analysis as many of them used non-parametric tests or different technologies 
were used in both experiment and control groups. Besides, some of the studies adopted qualitative data 
interpretation. 


As illustrated in Figure 1, initially 267 studies were identified and examined for their potential suitability to the 
current study considering the criteria noted above. While sorting and listing the selected studies, 113 of them 
were found to be multiple copies or duplicate studies indexed in more than one database, resulting in their 
exclusion. Afterward, the remaining 154 studies were screened by title and abstracts. As a result, 94 studies were 
excluded because they reported the impact of digital technology on other aspects of foreign languages other than 
vocabulary. The full-text of the remaining 60 studies underwent to further eligibility screening and ended up 
with the exclusion of 26 studies because five of them used non-parametric tests, 13 only had experimental 
groups all utilizing technology, two of them reported descriptive findings by giving percentages only, three of 
them employed qualitative methods, and three of them had limited access with no contact details to the authors. 
By the end, 34studies were found to be eligible for inclusion to the study. Figure | depicts the flowchart of the 
exclusion and inclusion processes of the studies obtained as a result of the literature search procedure. The 
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flowchart was adapted from Moher, Liberati, Tetzlaff, and Alman (2009) who have granted full permission for 
reproduction. 


Search result of studies indexed in different 
databases (N=267) 


YOK Thesis: (n=88) 
- ULAKBIM: (n=21) 
- EBSCO: (n=123) 

ERIC: (n=35) 


Indentifying 


After removing duplicates and 
multiple copies (n=154) 


Screening 


After screening titles and abstracts Excluded literature 
n=154 n=94 


Excluded studies according to the criteria 
set: (n=26) 

Non-parametric (n=5) 

Only Experimental Groups, all utilized 
Technology (n=13) 

Descriptive studies, used only percentages 
(n=2) 

Qualitative Studies (n=3) 
Full-text inaccessible(n= 3) 


Full-text suitability check 
n=60 


Eligibility 


Inclusion 


Eligible to the Meta-Analysis 
n=34 


Figure 1The inclusion and exclusion process of the studies (see Moher et al, 2009) 


2.4. Quality Appraisal of the Selected Studies 


After identifying the eligible studies for the current study, they were further undertaken for quality check 
adopting a quality scoring system suggested by Pulye, Gagnon, Griffiths, and Johnson-Lafleur (2009). This 
quality appraisal system employs a dichotomous binary scoring technique by giving ‘1’ to the study if a quality 
indicator is met and ‘0’ if not met. However, in this study, a score of ‘0.5’ was added for studies that partially 
met the pre-defined quality indicators. Then, the overall quality score percentage for each study was computed. 
First of all, the quality indicators were determined by setting three—item criterion that reflects the nature of the 
experimental studies as recommended by Pulye et al (2009, p. 550), that is: 


1) The study reports the research implementation procedure, uses a random sampling technique for 
assigning the subjects to the treatment and control groups and controls the extraneous variable(s). 

2) Conceals the subject allocation information of the groups (i.e. the presence of blinding technique). 

3) The validity and reliability of the data are ensured and the number of subject withdrawal during the 
treatment is reported if there is any. 
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To compute the quality score percentage, the sum of the criteria score was divided by the total number of the 
criteria set (3 in this study) multiplied by 100. A study that received a quality score of 50% or more was 
considered of good quality whilst under that threshold (< 50%) was considered low quality. However, none of 
the studies were excluded because of their low quality, but their effect sizes were compared to that of quality 
studies. The quality appraisal procedure was carried out independently by at least two researchers and the results 
were compared to ensure their validity and reliability. As a result, 12 studies scored lower than 50% falling into 
the low quality study category according to the three criteria given above. 


2.5. Coding of Study Characteristics 


The studies that met the inclusion and exclusion criteria were coded based on the author names, the publication 
year, the study type (master or doctoral theses and proceedings), the duration time of the experiment, and the 
education level. Besides, the sample size of the treatment and control group, their mean post-test scores (if there 
is not a significant difference between pre-tests of treatment and control groups) and SDs were coded.Lipsey and 
Wilson (2001, p.86) recommended to code 20 or more studies for a consistent reliability estimate between the 
coders. Therefore, 20 randomly selected studies were included in the meta-analysis and two coders were 
requested to fill the coding form in order to determine the reliability of the coding. For reasonable reliability, the 
agreement rate should be more than 80 percent (Miles & Huberman, 1994). The inter-coder reliability has been 
calculated as 100 percent through the use of formula as follows: 


Number of observations agreed upon 
Agreementrate = @$-[TAAANNIIINM____ 
Total number of observations 


2.6. Data Analytic Strategy 


This meta-analytic review utilized Hedges g as an effect size index with a view to calculating the effect of digital 
technology-assisted teaching on vocabulary learning. Hedges g refers to the difference between the two groups 
mean in terms of general standard deviation and is more suitable for the sample lower than 20 (Cooper, 2010, pp. 
163-168). Hedges’ g was used in the present study since the sample size of some primary studies included in this 
meta-analysis was lower than 20. The effect sizes are classified and interpreted as: 0-0.20 = very small, 0.21- 
0.50 = small, 0.51-1 = medium, and >1.0 = large effect (Cohen, Manion, & Morrison, 2007, p. 521). 


In the meta-analysis method, the effect size is calculated by combining these effect sizes according to the fixed 
or random effect models after calculating the effect sizes of each study. A test for heterogeneity is carried out in 
order to determine which model can be used to combine the effect sizes of the studies. In fact, the random effects 
model is recommended provided that the studies are obtained from the relevant literature (Borenstein et al., 
2013, p.86). Thus, the general effect size has been identified according to the random effects model. Besides, a 
heterogeneity test has been used to determine the presence and magnitude of heterogeneity within studies. 
Heterogeneity tests examine as to whether the observed variance in effect sizes varies across the expected 
variance due to sampling error (Cooper, 2010, p. 185). Hence, this research has examined Q-value or the 
significance level of the heterogeneity test (p< .05). However, Q statistics do not refer to the level of 
heterogeneity of studies (Card, 2012, p.188). At that point, 7? index is used. The 7 index is interpreted as 
follows: up to 25% ‘low’, up to 50% ‘medium’ and up to 75% ‘high’ heterogeneity (Higgins, Thompson, Deeks 
and Altman, 2003). 


This research has also performed categorical and regression moderator analysis for the purpose of identifying 
whether the common effect size of the digital technology-assisted teaching on vocabulary learning significantly 
varies across study types, study quality, education level, grouping methods (random or non-random assignment), 
technology types and treatment duration. Besides, a regression moderator analysis was carried out on treatment 
duration in the individual studies in order to explore if amount of exposure to technology-aided instruction 
makes any difference in students vocabulary learning. 


Publication bias indicates the likelihood that a group of studies selected from published studies on a particular 
topic may not represent all studies (Rothstein, Sutton, & Borenstein, 2006). If the studies that are statistically 
significant are mostly examined in a meta-analysis, this analysis is likely to have publication bias (Borenstein et 
al., 2009). The funnel diagram, Duval and Tweedie’s Trim and Fill and the Rosenthal’s Fail-Safe N Test have 
been used to determine the publication bias of the studies and its effect on the obtained results. Meta-analysis has 
been conducted through the use of Office programs and CMA 2.0 trial version and R metafor (Viechtbauer, 
2017). 
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3. FINDINGS 


This meta-analytic study was conducted aiming to combine the sixteen years of experimental research findings 
in Turkey on the impact of digital technologies on foreign language vocabulary learning. After following the 
procedures required for the meta-analytic studies elaborated in the methods section, the first attempt was to 
compute the effect sizes. Since the primary studies were obtained from the existing literature, a random effect 
model was preferred (Borenstein et al., 2013). The Forest Plot given in Figure 2 illustrates the resultant effect 
sizes with accompanying statistics: 
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Figure 2 Forest plot displaying the effect sizes and 95% confidence interval 


As seen in Figure 2, the majority of the effect sizes (n = 32, 78.05%) are significant, while only a few of them 
are not (n = 9, 21.95%). Besides, more than half of the effect sizes were found large (n = 20, 48.78%), some 
medium (” = 12, 29.27%) while a few (n = 9, 21.95%) either small or very small across the individual studies 
and their subgroups. The proportional figures at the 95% CI level illuminate that most of the effect sizes are in 
favour of the treatment groups. 


Also, combining a total of 41 effect sizes derived from 34 studies selected for this study yielded a common 
random effect size of g = 1.173 with 95% CI of 1.429 and 0.918. This result shows a strong effect according to 
the effect size classification criteria (Cohen et al., 2007, p. 521). Therefore, it can be concluded that the 
incorporation of digital technologies in foreign language teaching can have a positive impact on students’ 
vocabulary learning achievement. 
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3.1. Heterogeneity Test and Moderator Analysis 


A heterogeneity test was run in order to find out the level of heterogeneity of the effect sizes computed. Though, 
the distribution of the effect sizes and their heterogeneity across the existing research findings can also be seen in 
the Forest plot given in Figure 2. 


Table 1. 
Heterogeneity Test 
> . 95% CI Heterogeneity 
uci & Hscen 2: pub EaO! “Teeter 0: ar 
Random Effect 41 1.173 0.130 0.917 1.428 308.378 40 0.000 87.029 


As seen in Table 1, Q value equals to 308.378 with 40 degrees of freedom yielding a significant result (p < 0.05) 
in terms of the heterogeneity of the studies considered for the current study. Comparison of the Q statistics to the 
table of chi-square critical values with 40 degrees of freedom considering the 95% Confidence Intervals (CI) 
indicated that the Q is overlapping x’scores (Cooper, Hedges & Valentine, 2009, p. 593). Besides, the ? index 
was found to be 87.03% showing a high level of variability and differences amongst the effect sizes obtained 
from the primary research findings. This means that the real effect size may contrast from one study to another. 


In order to trace the heterogeneity sources of the effect sizes noted above, several moderator analyses were 
conducted. Study type, study quality, education level, grouping method (random & non-random assignment), 
and the technology type used in the treatment group were considered as categorical moderators as portrayed in 
Table 2. However, the treatment duration in the studies was evaluated through a regression moderator analysis in 
order to discover if it is a meaningful predictor of increased vocabulary learning via digital technologies. 


Table 2. 
Moderator Analysis 
Categorical Moderator Effect Size 95% Confidence Intervals Zp) Heterogeneity 
N Lower Limit _ Upper Limit OQ, df op 
Study Type 41 1.007 0.794 1.220 0.000 5.955 1 0.015 
Article 12 = 0.802 0.532 1.071 0.000 
Thesis 29 1.349 1.002 1.697 0.000 
Study Quality 41 1.187 0.934 1.440 0.000 0.235 1 0.628 
High Quality 29 1.219 0.936 1.502 0.000 
Low Quality 12 1.063 0.500 1.626 0.000 
Education Level 41 1.051 0.837 1.265 0.000 2.090 3 0.554 
Higher Education 28 1.157 0.860 1.455 0.000 
Pre-school 3 2.068 -0.486 4.623 0.113 
Primary 3 0.850 0.453 1.247 0.000 
Secondary 7 1.032 0.534 1.529 0.000 
Grouping Method 410.912 0.727 1.097 0.000 9.716 1 0.002 
Random 16 = 0.756 0.547 0.965 0.000 
Not Random 25 1.467 1.072 1.861 0.000 
Type of Technology 34 1.053 0.799 1.308 0.000 3.514 2 0.173 
Computer 16 0.998 0.578 1.418 0.000 
Internet 7 1.788 0.977 2.600 0.000 
Mobile 11 0.956 0.608 1.305 0.000 


As seen in Table 2, the calculated effect sizes of only two moderators, study type and grouping methods, were 
found meaningful (Q,>x?; p< 0.05 and Q,>x?; p< 0.05 respectively). Considering the sub-category of study 
types, the common effect size with reference to the theses was found larger (g = 1.349; 95% CI = 1.697 & 1.002) 
than that of the articles (g = 0.802; 95% CI = 1.071 & 0.532). Similarly, the non-random grouping effect size 
was found larger (g = 1.467; 95% CI = 1.861 & 1.072) than the random one (g = 0.756; 95% CI = 0.965 & 
0.547). To sum up, the heterogeneity of the effect sizes is seemingly rooted in the study types (i.e. articles and 
theses) and the grouping methods (i.e. random & non-random assignment). However, the other moderators were 
found insignificant with medium to large effect sizes (Q,<x; p> 0.05). Likewise, the results of the mixed-effects 
regression moderator analysis (given in Table 3) indicated that the duration of experiments in the primary studies 
is not a meaningful predictor of technology-assisted foreign language vocabulary learning achievement (Q = 
0.50, df= 1, p> 0,05). 
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Table 3. 

Regression moderator analysis for the ‘Treatment Duration’ in the primary studies 

Method of Moments Q df p 
Model 0.49841 1 0.48020 
Residual 57.20712 39 0.03005 
Total 57.70553 40 0.03456 
3.2. Publication Bias 


Majority of the effect sizes reported in the previous sections were found large. However, in order to reveal 
whether they are representing real effect sizes or just produced because of the potential publication bias, several 
evaluations have been made despite the fact that majority of the effects sizes computed are related to the 
unpublished theses (n = 22, 70.73%) compared to the articles (n = 12, 29.27%). The 29 effect sizes computed for 
22 theses were found larger (g = 1.349) than that of the articles (g = 0.802). This can reflect the validity of the 
computed effects sizes. However, when the funnel plot in Figure 3 was evaluated, the individual effect sizes 
were asymmetrically distributed around the mean effect size pointing that there is possible publication bias. 
Symmetric distribution is expected when the effect sizes are positioned in close proximity around the mean 
estimate to assume that the bias does not exist (Borenstein et al., 2009). 


0 


0.126 


Standard Error 
0.378 0.252 


0.504 


Observed Outcome 


Figure 3Funnel Plot Showing Asymmetric Distribution of Effect Sizes 


However, further evaluations were made via Duval and Tweedie’s Trim and Fill technique in order to reveal if 
the asymmetric effect size distribution displayed in the Funnel plot is true. As a result, it was found that the 
observed effect size (g = 1.17295; CI = 1.42845, 0.91746) is not comparatively different from that of adjusted 
one (g = 1.41738; CI = 1.13250, 1.70226). Both of these effect sizes show that these “results are robust to 
publication bias” since both of “the estimates are comparable” (Card, 2012, p. 274). Therefore, the asymmetry 
displayed in the Funnel plot could be falsified since it causes subjective interpretation according to Borenstein et 
al (2009). See Table 4 for more details. 


Table 4. 

Duval and Tweedie’s Trim and Fill 

Random effect model _ Trimmed Studies Hedge’s g Lower limit Upper limit Q value 
Observed values 1.17295 0.91746 1.42845 308.37757 
Adjusted values 7 1.41738 1.13250 1.70226 550.41677 


In addition, Rosenthal’s Fail-Safe N test was run in order to ensure that the common effect size is true and not 
resulted from the publication bias. The results indicated that 5465 studies with the common effect of zero are 
required to nullify the resultant common effect size in this study. This can also prove that the computed effect 
sizes are unbiased by supporting the findings above. According to Borenstein et al (2009), it will be a matter of 
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concern if “only a few studies (say, five or ten)” cause effect size nullification (p. 284). However, if a big 
number of studies were missing to invalidate the effect, it will be of little concern. 


4. DISCUSSION AND CONCLUSION 


This research combined the results of 34 experimental studies which were carried out in Turkey during 2002- 
2018 academic years and which explored the impact of digital technology-mediated foreign language instruction 
on vocabulary learning through a meta-analytical review method. The common effect size was identified as 
1.173 depending on random effects model, meaning that the integration of digital technologies in foreign 
language teaching may have a profound positive impact (g = 1.173) on students’ vocabulary learning 
achievement. Likewise, Abraham (2008) presented findings from a meta-analysis of 11 studies of computer- 
mediated glosses in second language reading comprehension and vocabulary learning. Accordingly, he found 
that computer-mediated glosses had a large effect on incidental vocabulary learning. This result is consistent 
with that of the current study. However, in the study conducted by Mahdi (2017), the findings of the meta- 
analysis indicated a medium impact of using mobile devices on vocabulary learning (g =.67). 


As indicated by Baytak, Tarman, and Ayas (2011) learners’ learning is improved by incorporating technology 
into the classroom. Numerous individual studies were conducted on the effectiveness of diverse digital 
technologies in foreign language vocabulary learning and acquisition and its superiority to traditional approaches 
in Turkey and abroad (Yip & Kwan, 2006; Nejati, Jahangiri, & Salehi, 2018; Fidan, 2003; Eren, 2015; Giivendir 
& Gezgin, 2015; Ozer & Kocoglu, 2017). In contrast, there existed some contradictory findings, indicating that 
no significant difference was noted between the instruction with or without technology in foreign language 
vocabulary learning (Toka¢, 2005; Erkus, 2008; Baséz & Can, 2016). 


The present study also analyzed whether the impact digital technology-mediated foreign language instruction on 
vocabulary learning differed across study type (thesis-article). As a result of the categorical moderator analysis, 
the calculated effect size of the moderator was found to be significant (p < .05). Considering the sub-category of 
study types, the common effect size with reference to the theses was found larger than that of the articles. The 
articles generated an effect size of (g=0.802), whereas, theses produced the effect size of (g=1.349), 
concluding that there is no publication bias in the resultant effect sizes. Li (2010) noted that published studies did 
not show a larger effect than Ph.D. dissertations. By the same token, the mean effect size for dissertations was 
found larger compared to the published articles. In contrast, different result has emerged in the study conducted 
by Kanadli, Unal & Karakus (2015). Card, Stucky, Sawalani, and Little (2008) noted that published studies 
(articles) are more likely to have a significant effect than unpublished studies (theses). Therefore, it is 
recommended to include both published and unpublished studies in order to prevent publication bias and to carry 
out diagnostic analyzes to determine if publication bias is present in meta-analysis studies. Thus, this study 
performed Duval and Tweedie trimming and filling statistics and Rosenthal’s Fail-Safe N, suggesting that this 
study was free from any publication bias. 


Another result of the present study showed that the impact of digital technology-mediated foreign language 
instruction on vocabulary learning did not vary across study quality. In other words, the study quality moderator 
was found insignificant with medium to large effect sizes. This paves the way for the fact that the studies with 
high and low quality are not an indicator that determines the effectiveness of digital technology-mediated foreign 
language teaching on vocabulary learning. Although the primary studies were categorized as low or high quality 
in terms of the methodology used, it does not mean that their results are invalid or the kind of technology used is 
not influencing students’ learning. Both categories displayed large effect sizes. 


Upon examining the impact of digital technology-mediated foreign language instruction on vocabulary learning, 
no significant difference has been noted across education level (Higher education, pre-school, primary, and 
secondary). In other words, digital technology-mediated foreign language instruction positively affects students’ 
vocabulary learning process irrespective to education level. In addition, research findings pointed out that the 
impact of digital technology-mediated foreign language instruction on vocabulary learning does not demonstrate 
a significant difference in terms of technology type (computer, internet, mobile). This opens the gate for the fact 
that digital technology-mediated foreign language instruction is efficient in vocabulary learning process no 
matter what technology type is used. The common effect sizes computed for computer and internet were large 
(more than 0.80) and that of mobile was medium (between 0.50 and 0.80). The use of computer, internet, and 
mobiles may be said to provide extra opportunities to the learners in order to practice language skills and 
facilitate vocabulary enrichment in the target language (Richards, 2015). 


This study also illustrated that the impact of digital technology-mediated foreign language instruction on 
vocabulary learning significantly varied across the grouping method (random, not random). The random 
grouping generated effect size of (g=0.756), whereas, not random grouping produced the effect size of 
(g = 1.467). This result shows that the common effect size of studies with non-random grouping on vocabulary 
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learning is large, while that of the studies with random grouping is low, yet this difference was found to be 
insignificant as a result of the moderator analysis. The main difference between these two groupings is that a 
random assignment is valid in the true-experimental design, whereas non-random assignment in quasi- 
experimental design (Biiyiik6ztirk et al., 2008, pp. 142-156). Thus, this difference can be welcomed since the 
present study mostly includes quasi-experimental studies. 


Lastly, the results of the mixed-effects regression moderator analysis indicated that the duration of experiments 
in the primary studies is not a significant predictor of technology-assisted foreign language vocabulary learning 
achievement. Under normal circumstances, treatment duration is a significant variable in foreign language 
vocabulary learning due to its novelty effect. To illustrate, in Kao’s study (2014), the treatment duration was 
found to be influential in vocabulary learning, accordingly, digital games were effective for the EFL learners 
who received the long treatment duration of education. This result is inconsistent with that of the present study. 


Recommendations 


This study revealed that in-class utilization of digital technology in foreign language instruction can substantially 
contribute to the students’ vocabulary development in the target language. This may, in turn, give students an 
inspiration in how to use or select the technological resources independent of the classroom context. Therefore, 
to provide effective learning opportunities for the students, the language teachers are recommended to try a 
variety of technological resources in the class. It does not require a classroom fully equipped with high-tech 
materials. A simple smart-phone or computer will do. However, teachers might need some methodological 
training about educational technology and the know-how of locating and using cost-effective resources in an 
efficient way. There is a bulk of well-documented research evidence about the use of educational technology in 
the field of foreign language education and its effectiveness is proven. Therefore, further empirical and meta- 
analytical studies could be recommended regarding the use of digital technologies in other fields of education to 
see how they impact the teaching and learning process in different fields. 


Limitations 


This meta-analytical review was limited to the studies conducted in Turkey in 2002—2018, and those accessed 
through the electronic media. Hence, the studies conducted abroad and published out of the electronic media 
were not included in this study. Likewise, the review includes only individual studies having experimental 
research design by ignoring those designed with relational survey model. Taking these limitations into 
consideration while conducting meta-analytical studies, it is expected to make a great contribution to the relevant 
literature. 
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GENIS OZET 


1. Giris 


Kelime 6grenimi, uzun yillardir yabanci dil 6greniminin temel ve 6nemli bir pargasi olarak kabul edilmistir. 
Cesitli arastrrmacilar, kelime 6greniminin yabanci ve ikinci dil yeterliliginin temel bir bileseni oldugunu 
belirtmektedir (Bozavli, 2017; Cellat, 2008; Harley, 1996; Nation, 2001; Read, 2000). Kelime 6grenme, 
teknoloji tabanli ve geleneksel olmak iizere iki farkli ortamda ortaya gikabilir (Mehdi, 2018). Teknoloji tabanli 
ortam, cevrimici ve cevrimdisi s6zliikler, multimedya, sosyal medya, televizyon, bilgisayar, akill telefonlar, 
tabletler ve mobil cihazlar gibi kelime 6grenme igin yeni ve modern teknolojilerin kullanilmasina isaret eder. Bu 
nedenle, hizli bir bigimde gelisen dijital teknolojinin, kelime 6grenen 6grencilere sinirsiz imkanlar sagladigini 
sdylemek mimkiindir. Ayrica, giiniimiizde dijital teknoloji, diger alanlarda oldugu gibi yabanci dil egitim 
sisteminin cesitlendirilmesinde olduk¢a 6nemi bir role sahiptir. 


Yabanci dilde kelime 6greniminde ¢esitli dijital teknolojilerin etkililigini ve egitimin ilk yillarndan tiniversiteye 
kadar farkl yas gruplari ve simif diizeylerini dikkate alarak geleneksel yaklasimlara gére avantajlarim incelemek 
amactyla Tiirkiye’de ve dtinyada pek cok deneysel ve betimsel calismalar yapilmistir (Yip & Kwan, 2006; 
Fageeh, 2013; Shoaei & Alavi, 2016; Nejati, Jahangiri, & Salehi, 2018; Fidan, 2003; Eren, 2015; Giivendir & 
Gezgin, 2015; Kurt & Bensen, 2017; Ozer & Kocoglu, 2017). Buna karsin, dijital teknoloji temelli 68retimin 
yabanci d,l kelime 6grenimi tizerinde herhangi bir etkisi olmadigini gésteren calismalar da mevcuttur (Tokag, 
2005; Erkus, 2008; Basoz & Can, 2016). Mevcut ¢alismalardaki bu farklilasmalan géz 6ntinde bulundurarak, 
dijital teknolojilerin Tiirkiye’de yabanci dil 6greniminde ne kadar etkili olduguna dair genel bir yargirya varmak 
amaciyla sentetik bir analiz yapilmasi gerekmektedir. Ulusal alanyazin incelendiginde, bilgisayar teknolojierinin 
yabanci dil kelime 6gretimi tzerindeki etkisini inceleyen sadece bir tane meta-analiz caligsmasi oldugu 
gértilmektedir (Tomakin & Yegilkurt, 2013). Fakat, alanyazinda dijital teknoloji temelli 6gretimin kelime 
6Srenimi tizerindeki etkisini inceleyen deneysel ¢alismalarin meta-analiz calismasinin olmadig1 tespit edilmistir. 
Bu nedenle, bu ¢alismanin amaci dijital teknolojiye dayali 6Sretimin yabanci dil kelime 6grenimi tizerindeki 
etkisini inceleyen Tiirkiye’de 2003-2018 yillari arasinda yapilan deneysel galigsmalarin meta-analizini yaparak 
genel etki biiyiikliinti hesaplamak ve bu etki biiytikliigiiniin cesitli degiskenler acisindan anlamli bir bigimde 
farklilasma gésterip gdstermedigini belirlemektir. 


2. Yontem 


Bu arastirma, deneysel ¢alismalarin sonuclarini birlestirmek amaciyla meta-analitik tarama yontemi kullanmistir. 
Meta analitik tarama yontemi belirli bir konuda yapilan bireysel galismalarm sonuglarini birlestirerek 
sentezlemeyi ve dolayisiyla bireysel galigmalarin sinirliliklarini istatistiksel olarak azaltmayi hedefleyen bir 
yontemdir (Biiytkéztiirk vd., 2009). Cooper (1998) meta-analiz stirecinin bes asamadan olustugunu belirtmistir: 
(1) problem formiilasyonu, (ii) veri toplama, (iii) veri degerlendirmesi (dahil etme ve hari¢ tutma kriterleri), (iv) 
veri analizi ve yorumlama, ve (v) bulgularin sunulmas1. 


Mevcut literatiir taramasi Eyliil 2017 - Aralik 2018 arasinda asagidaki gibi cesitli veritabanlari kullanilarak 
gerceklestirilmistir, ERIC, ULAKBIM, EBSCO ve YOK Tez Arsivi. Son arama Ocak 2019'da yapilmistir. 
Arastirmalar, hem Tiirkce hem de ingilizce olarak cesitli anahtar terimler kullanilarak taranmistir: “kelime 
o6Srenme”, “kelime edinme ”,“ kelime 6grenme ve teknoloji ”,“ dijital teknoloji ve kelime 6grenme ”,“ dijital 
teknoloji destekli kelime 6grenme ”,“ dijital teknoloji temelli kelime 6gretimi ”,“‘ kelime gelisimi ”ve“ teknoloji 
ile kelime 6grenimi ”. Arastirma siireci iki arastirmaci tarafindan ger¢eklestirilmis ve arastirmacilar tarafindan 
bulunan g¢alismalarm benzer olup olmadigina karar verilerek ilgili ¢alismalar secilmistir. Sonug olarak, 
arastrmacilar, Tiirkiye'de dijital teknoloji destekli kelime 6grenme konusunda 267 calismaya ulasmuislardir. 
Ardindan, galismalari dahil etme kriteleri belirlenmis, ve bu dogrultuda cgaligmalar elenerek 34 ¢aligma ile analiz 
surecinde gidilmistir. 


Bu caligmada her bir calismanin etki bitytikliigiintin hesaplanmasinda Hedge’s g katsayisi kullanilmistir. Her bir 
calismanin etki biiyiikligii hesaplandiktan sonra genel etkinin hangi modele (sabit ya da rastgele etkiler modeli) 
dahil edilerek hesaplanacagini belirlemek icin heterojenlik testi yapilmistir. Hesaplanan genel etki bitytikligiintin 
calisma tirtine, 6gretim kademesine, calismanin kalitesine, gruplama yéntemine ve kullanilan teknoloji tiirtine 
g6ére anlamli bir farklilik gésterip géstermedigini belirlemek icin kategorik moderatér analizi yapilmistir. Bunun 
yan sira, bu etkinin uygulama siiresine gére anlamli bir bigimde farklilasma gésterip géstermedigine regresyon 
moderatér analizi yapilmistir. Yayin yanliliginmn varligini belirlemek ve analiz tizerindeki etkisini 
degerlendirmek amaciyla diagnostik analiz gerceklestirilmistir. Diagnostik analizde huni grafigi, Duval ve 
Tweedie’nin kirpma ve doldurma istatistigi, Rosenthal’1n Korumali N testi kullanilmistir. 
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3. Bulgular, Tartisma ve Sonuc 


Bu arastirma, 2002-2018 egitim-dgretim yilinda Tiirkiye'de gercgeklestirilen ve dijital teknoloji temelli yabanci1 
dil 6gretiminin kelime 6grenimi tizerindeki etkisini bir meta-analitik inceleme y6ntemi ile inceleyen 34 deneysel 
caligsmanin sonuclar birlestirilerek yiritiilmiistiir. Arastirma sonucunda, yaygin etki modeline bagli olarak ortak 
etki biiyiikligii 1.173 olarak belirlenmistir. Bu durum, dijital teknolojilerin yabanci dil 6gretimine 
entegrasyonunun 6grencilerin kelime 6grenme basarisi tizerinde genis capta olumlu bir etkiye sahip oldugunu 
gdstermektedir. Benzer sekilde, Abraham (2008) bilgisayar kullaniminin kelime 6grenimi iizerinde biyiik bir 
etkisi oldugunu ortaya koymustur. Bu sonu¢ mevcut calismanin sonucuyla tutarlilik géstermektedir. 


Arastirmada, dijital teknoloji temelli yabanci dil 6gretiminin kelime 6grenimi tizerindeki etkisinin galisma tiirtine 
(tez-makale) gére anlamli farklilik gésterip géstermedigini belirlenmistir. Yapilan kategorik moderatér analizi 
sonucunda, hesaplanan etki biiyiikligtintin calismanin tiirtine gére anlamli bir bicimde farklilastig1 gériilmektedir 
(p <.05). Calisma tiirlerinin alt kategorisi g6z 6ntine alindiginda, bu farklilasmanin tezlerin lehine oldugu tespit 
edilmistir. Li (2010) tarafindan yapilan ¢alismada, makalelerin doktora tezlerinden daha biiyiik bir etki 
géstermedigi sonucuna ulasilmistir. Bu sonug mevcut ¢alismanin sonuclartyla tutarlilik géstermektedir. Buna 
karsin, Kanadh, Unal & Karakus (2015) yaptiklar calismada etki biiyiikligiiniin makale tiiriindeki calismalar 
lehine oldugu belirlenmistir. 


Bu calismanin bir diger sonucu, dijital teknoloji temelli yabanci dil 6gretiminin kelime 6grenimi tizerindeki 
etkisi calisma kalitesine gére anlamli bir bigimde farklilasmamasidir. Bu durum, yiiksek ve diisiik kalitede 
yapilan galismalarin, dijital teknoloji temelli yabanci dil 6gretiminin kelime 6grenimindeki etkililigini belirleyen 
bir gésterge olmadigi gergegini ortaya koymaktadir. Ayrica, dijital teknoloji temelli yabanci dil 6gretiminin 
kelime 6grenimi tzerindeki etkisinin 6gretim kademesi ve kullanilan teknoloji tiiriine gére de anlamli farklilik 
géstermedigi sonucuna ulasilmistir. Fakat, yapilan analizler sonucunda, dijital teknoloji temelli yabanci dil 
OSretiminin, 6gretim kademesine bakilmaksizin 6grencilerin kelime 6grenme stirecini olumlu yénde etkiledigi 
gértilmektedir. Bunun yaninda, etki biiyiikliigti kullanilan teknoloji tiirtine gére anlamli bir bicimde farklilik 
gdstermese de, dijital teknoloji temelli yabanci dil 6gretiminin kelime 6grenme stirecinde etkili oldugu ifade 
edilebilir. Bilgisayar ve internet i¢gin hesaplanan etki biiyiikliiklerinin genis gapta (0.80'den fazla) ve mobilin orta 
diizeyde (0.50 ile 0.80 arasinda) oldugu bu durumu kanitlar niteliktedir. Bilgisayar, internet ve cep telefonlarinin 
kullanilmasinin, 6grencilerin dil becerilerini pekistirmek ve hedef dilde kelime zenginligini kolaylastirmak icin 
ek firsatlar sagladigi s6ylenebilir (Richards, 2015). 


Arastirma sonucunda, dijital teknoloji temelli yabanci dil 6gretiminin kelime 6grenimi tizerindeki etkisinin 
gruplama y6ntemine gore rastgele gruplama lehine oldugu sonucuna ulasilmistir. Son olarak, regresyon 
moderator analizinin sonuclari, birincil ¢alismalarda uygulama siiresinin, teknoloji destekli yabanci dil kelime 
6renme basarisinin 6nemli bir géstergesi olmadigini ortaya koymaktadir. Normal sartlar altinda, uygulama 
stiresi, inovasyon etkisinden dolay1 yabanci dil kelime 6greniminde 6nemli bir degiskendir. Bu sonucun aksine, 
Kao (2014) galismasinda kelime 6greniminde uygulama stiresinin etkili oldugunu belirlemistir. 


Yapilan bu calisma ile dijital teknolojinin simif iginde yabanci dil 6gretiminde kullanilmasinin, 6grencilerin 
hedef dilde kelime hazinelerinin gelismesine 6nemli 6l¢tide katkida bulunabilecegi ortaya konmustur. 
Dolayistyla, 6grencilere etkili 6grenme firsatlar1 saSlamak icin, yabanci dil 6gretmenlerine simifta ¢esitli 
teknolojik kaynaklari kullanmalar Gnerilmektedir. Bununla birlikte, 6gretmenlere egitim teknolojisi ve diisiik 
maliyetli kaynaklari verimli bir sekilde bulma ve kullanma konusunda egitimler verilmesi 6nerilmektedir. 
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